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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
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Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 
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3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1, 453 O.G. 213. 
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PROSECUTION IS HEREBY REOPENED 

In view of the Appeal Brief filed on December 22, 2004, PROSECUTION IS HEREBY 
REOPENED. New ground of rejection set forth below. 

To avoid abandonment of the application, appellant must exercise one of the following 
two options: 

(1) file a reply under 37 CFR 1 .1 1 1 (if this Office action is non-final) or a reply under 37 
CFR 1.1 13 (if this Office action is final); or, 

(2) request reinstatement of the appeal. 

If reinstatement of the appeal is requested, such request must be accompanied by a ' 
supplemental appeal brief, but no new amendments, affidavits (37 CFR 1.130, 1.131 or 1.132) or 
other evidence are permitted. See 37 CFR 1.193(b)(2). 

Claim Rejections - 35 USC § 112 

The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

Claims 1-27 rejected under 35 U.S.C. 112, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. Applicant in the preamble of the independent claims uses a term " graph 
rewriting". Examiner's comment: Graph is a drawing illustrating the relations between certain 
quantities. Applicant requires to explicitly specifying the meaning of the term "graph" 
corresponding to the term "rewriting". Should it be written as "graph redrawing"? 
Does the term "multimedia content" contain an image, sound, video, text and etc.? 
What are the pre-defined templates and where are they stored? 
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What method of comparing does the Applicant use? 

What does occur to "validating the input graph" if there is no match with a template graph? 
Dose Applicant's computerized method analyze the content of multimedia prior to storage or 
after? 

Is the graph rewriting exactly similar to the multimedia content? 

Does the comparing scenario identify all objects for multimedia content? 

Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

Claims 1-27 rejected under 35 U.S.C. 102(b) as being anticipated by Euripides G.M. 
Petrakis, Member, IEEE, and Christos Faloutsos (hereafter refers as a "Euripides"). 
* Claim 1. 

Euripides discloses on page 435 under section 1, that shows a computerized method, quote 
from the reference: In medicine, in particular, a large number of images of various imaging 
modalities (e.g., computer tomography, magnetic resonance, etc.) are produced daily and used 
to support clinical decision making. A computerized method of graph rewriting, a graph having 
nodes representing entities and edges between the nodes representing relationships between 
entities, the method comprising: Euripides in figs. 1-2 illustrates graph rewriting of an image 
and its corresponding ARG (Le. graph rewriting). Euripides on page 436 under section 2.1 
discloses image descriptions are given in terms of object properties, and in terms of 
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relationships between objects. The textbook approach to capture this information is the 
Attributed Relational Graphs (ARGs). In an ARG, graph nodes represent the objects, and arcs 
between such nodes represent the relationships between objects. Both nodes and arcs are 
labeled by attributes corresponding to properties (features) of objects and relationships, 
respectively. 

As claim invention discloses that comparing an input graph representing a description scheme 
for multimedia content with a set of predefined template graphs. Euripides on page 437 in first 
column and in fig. 2 discloses the specific features, which are used in ARGs, are derived from 
the raw image data and, depending on the application, can be geometric (Le., independent of 
pixel values), statistical, or textural, or features specified in some transform domain (e.g., 
Fourier coefficients of object shapes). In the case of medical CT and MRI images used in this 
work, the set of features is given in Section 4.1. However, the proposed methodology is 
independent of any specific kind of features. The problem of retrieving images, which are 
similar to a given example, image is transformed into a problem of searching a database of 
stored ARGs: Given a query, its ARG has to be computed and compared with all stored ARGs. 
Matching between ARGs is a well-known problem and has been studied extensively in the 
computer vision literature. Specifically, matching a query and a stored graph is treated as a 
sub graph isomorphism problem. 

As claim invention discloses in the last line of claim one that validating the input graph when 
there is a match with a template graph. Euripides on page 437 in second column discloses that 
the first term in Eq. (1) is the cost of matching (validating as the claim language specifies) 
associated nodes, while the second term is the cost of matching the relationships between such 



4 



Application/Control Number: 1 0/044, 1 68 Page 5 

Art Unit: 2672 

nodes. In our setting, only a subset of the objects in the stored image S needs to be matched. 
There is no cost if the data image contains extra objects; however, we assume that the cost is 
infinite if the data image is missing one of the objects of the query. COST is the cost of 
matching features of objects or features of relationships between associated objects. The 
distance between images Q and S is defined as the minimum distance computed over all 
possible mappings FQ: see Eqs. 2-4. Similarity searching in an IDB of stored ARGs requires 
that all images within distance t must be retrieved. Specifically, we have to retrieve all the 
images S that satisfy the Eq. 4 condition. 
Claim 2. 

The computerized method of claim 1, wherein the comparing uses a graph matching 
process. Euripides on page 436 under section 2.1 discloses image descriptions are given in 
terms of object properties, and in terms of relationships between objects. The textbook 
approach to capture this information is the Attributed Relational Graphs (ARGs). In an ARG, 
graph nodes represent the objects, and arcs between such nodes represent the relationships 
between objects. Both nodes and arcs are labeled by attributes corresponding to properties 
(features) of objects and relationships, respectively. 

Claim 3. 

The computerized method of claim 2, wherein the comparing comprises: creating 
adjacency matrices representing the input graph and the set of template graphs. 
Euripides on page 440 under section 4.1 and in second column teaches that additional 
features that could be used include the average gray-level and texture values, moments, or 
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Fourier coefficients, etc., as object descriptors; relative size, amount of overlap-ping, or 
adjacency, etc., can be also used to characterize the relationships between objects. 
Claim 4. 

The computerized method of claim 1 further comprising: evaluating the input graph against a set 
of pre-defined alphabet graphs; and applying a rule associated with a matching alphabet graph to 
the input graph, the rule represented by a rule graph and a set of morphism graphs. Euripides on 
page 437 in column one discloses matching a query and a stored graph is treated as a sub 
graph isomorphism problem. Examiner comment: the meaning of morphism is an abstraction 
of a function or mapping between two objects. 
Claim 5. 

The computerized method of claim 4, wherein the evaluating uses a graph matching 
process. This step is inherent in the reference see figs. 1-3 of the Euripides. 
Claim 6. 

The computerized method of claim 5, wherein the evaluating comprises: creating 
adjacency matrices for the input graph and the set of alphabet graphs. Euripides on page 439 in 
the first column discloses that the CAFIIR system proposes the "iconic index tree" to 
accelerate the search on facial images. One novelty of the system is that it can process "fuzzy" 
(Le., subjective or in-complete) queries, through the so-called "fuzzification" technique, which 
translates the feature space to a fuzzy space. Also see page 440 the last paragraph before 
section 4.2 Euripides discloses set of alphabet graph or image translated or scaled with respect 
to each other. 

Claims 7 and 8. 
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The computerized method of claim 4, wherein the applying comprises: performing 
pushout/pullback operations. As Applicant on page 11 paragraph 0029 discloses that the two 
particular operations in algebraic graph grammars are suitable to build graph rewriting 
techniques for description scheme graphs: pushout and pullback Pushouts and pullbacks can be 
thought of as sums and products, respectively. Euripides on page 437 in Eqs. 1 -4 clearly shows 
the meaning of sums and products. Also in fig. lb illustrates the pullback (also called the fiber 
product) is the limit of a diagram consisting of at least two morphisms r23: v2 v3 and rl2: 

vl -> v3 with a common codomain (A set within which the values of a function lie (as opposed 

to the range, which is the set of values that the function actually takes). Explicitly, the 
pullback of the morphisms r23 and rl2 consists of an object vO and two morphisms rOl: vO 

vl and r02: vO -> v3 for which the diagram. The categorical dual of a pullback is a called a 
pushout. 

Claim 9. 

The computerized method of claim 8, wherein performing the pullback operation comprises: 
creating adjacency matrices representing smallest portions of the set of morphism graphs that 
map the input and rule graphs to the alphabet graph using pre-images of parts of the alphabet 
graph marked for change; and multiplying the adjacency matrix associated with the input graph 
by a transpose of the adjacency matrix associated with the rule graph. See rejection of claims 7-8 
that applies to the rejection of this claim. 
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Claim 10 
See rejection of claim 1. 
Claims 11-18 

The rejections of claims 2-9 apply to the rejections of claims 1 1-18. 
Claims 19-27 

The rejections of claims 2-9 apply to the rejections of claims 19-27. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Javid A Amini whose telephone number is 571-272-7654. The 
examiner can normally be reached on 8-4pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Michael Razavi can be reached on 571-272-7664. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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Examiner 
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